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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings of claims in the application: 
Listing of Claims: 

1 . (Previously Presented) A computer-implemented method of providing 
information to a user based upon contents of a first document displayed to the user, the method 
comprising: 

identifying at least a first section of the first document; 

extracting a first set of one or more information objects from the first section of 
the first document, the first set of information objects comprising at least a first information 
object comprising information of a first type, wherein the first information object is one of an 
audio object, a video object, or an image object; 

determining degree of relevancy information for a second set of one or more 
information objects, the degree of relevancy information indicating the relevancy of information 
objects in the second set of information objects to information objects in the first set of 
information objects, wherein the second set of information objects comprises at least one 
information object comprising information of a second type, wherein the second type is different 
from the first type, and wherein determining the degree of relevancy information comprises 
determining a degree of relevancy of the at least one information object to the first information 
object from the first set of information objects; and 

selecting a third set of one or more information objects from information objects 
in the second set of information objects based upon the degree of relevancy information 
determined for information objects in the second set of information objects, wherein information 
objects in the third set of information objects store information to be output to the user when the 
first document is being displayed to the user. 

2. (Previously Presented) The method of claim 1 wherein the first section of 
the first document corresponds to a section of the first document displayed to the user, wherein 
the section of the first document displayed to the user is less than the entire first document. 
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3. (Previously Presented) The method of claim 1 wherein the first section of 
the first document corresponds to the entire first document. 

4. (Previously Presented) The method of claim 1 wherein extracting the first 
set of information objects from the first section of the first document comprises: 

for each information object in the first set of information objects: 

identifying a type of the information object based upon contents of the 

information object; 

determining a first content recognition technique based upon the type of 
the information object; and 

applying the first content recognition technique to the information object 
to determine information related to the contents of the information object. 

5. (Previously Presented) The method of claim 1 wherein: 

determining the degree of relevancy information for the second set of information 
objects comprises: 

identifying a plurality of selection techniques for determining the degree 
of relevancy information; and 

for each selection technique in the plurality of selection techniques, 
applying the selection technique to generate relevancy scores for the information objects in the 
second set of information objects, the relevancy scores indicating the relevancy of information 
objects in the second set of information objects to information objects in the first set of 
information objects calculated using the selection technique; and 

selecting the third set of information objects comprises: 

selecting one or more information objects from the second set of 
information objects to be included in the third set of information objects based upon the 
relevancy scores for information objects in the second set of information objects calculated using 
the plurality of selection techniques. 
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6. (Previously Presented) The method of claim 5 wherein selecting the one 
or more information objects from the second set of information objects to be included in the third 
set of information objects based upon the relevancy scores for information objects in the second 
set of information objects calculated using the plurality of selection techniques comprises: 

for each information object in the second set of information objects: 

calculating an aggregate relevancy score for the information object by 

aggregating the relevancy scores generated for the information object by applying the plurality of 

selection techniques; and 

selecting the information object to be included in the third set of 

information objects if the aggregated relevancy score calculated for the information object is 

above a threshold value. 

7. (Previously Presented) The method of claim 1 wherein: 

determining the degree of relevancy information for the second set of information 
objects comprises: 

identifying a first selection technique and a second selection technique for 
determining the degree of relevancy information; and 

applying the first selection technique to generate a first set of relevancy 
scores for information objects in the second set of information objects, the first set of relevancy 
scores indicating the relevancy of information objects in the second set of information objects to 
information objects in the first set of information objects calculated using the first selection 
technique; 

applying the second selection technique to generate a second set of 
relevancy scores for information objects in the second set of information objects, the second set 
of relevancy scores indicating the relevancy of information objects in the second set of 
information objects to information objects in the first set of information objects calculated using 
the second selection technique; and 

selecting the third set of information objects comprises: 
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selecting one or more information objects from the second set of 
information objects to be included in the third set of information objects based upon the first set 
of relevancy scores and the second set of relevancy scores. 

8. (Previously Presented) The method of claim 7 wherein applying the first 
selection technique to generate the first set of relevancy scores comprises: 

determining a set of one or more concepts of interest to the user; 

determining relevancy of each information object in the first set of information 
objects to each concept in the set of concepts; 

determining relevancy of each information object in the second set of information 
objects to each concept in the set of concepts; and 

calculating the first set of relevancy scores based upon the relevancy of each 
information object in the first set of information objects to each concept in the set of concepts 
and based upon the relevancy of each information object in the second set of information objects 
to each concept in the set of concepts, wherein each relevancy score in the first set of relevancy 
scores indicates a degree of relevancy of an information object in the second set of information 
objects to an information object in the first set of information objects for a particular concept 
included in the set of concepts. 

9. (Previously Presented) The method of claim 7 wherein applying the 
second selection technique to generate the second set of relevancy scores comprises: 

for each information object in the first set of information objects: 

identifying a type of the information object based upon contents of the 

information object; 

determining a comparison technique based upon the type of the 
information object; and 

for each information object in the second set of information objects, 
applying the comparison technique to generate a relevancy score for the information object in the 
second set of information objects, the relevancy score indicating a degree of relevance of the 
information object in the second set of information objects to the information object in the first 
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set of information objects using the comparison technique determined based upon the type of the 
information object in the first set of information objects. 

10. (Previously Presented) The method of claim 1 further comprising 
communicating the third set of information objects to a user system which is used to output 
information stored by information objects in the third set of information objects to the user. 

11. (Previously Presented) The method of claim 1 wherein the first document 
is displayed to the user using an access program and the information stored by information 
objects in the third set of information objects is output to the user in a predetermined area of the 
access program. 

12. (Previously Presented) The method of claim 1 1 wherein the access 
program is a web browser and the first document is a web page. 

13. (Previously Presented) The method of claim 1 further comprising: 
determining when a second document is displayed to the user instead of the first 

document; 

identifying at least a first section of the second document; 

extracting a fourth set of one or more information objects from the first section of 
the second document; 

determining new degree of relevancy information for the second set of 
information objects, the new degree of relevancy information indicating the relevancy of 
information objects in the second set of information objects to information objects in the fourth 
set of information objects; and 

selecting a fifth set of one or more information objects from information objects 
in the second set of information objects based upon the new degree of relevancy information 
determined for the second set of information objects, wherein information objects in the fifth set 
of information objects store information to be output to the user when the second document is 
being displayed to the user. 
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14. (Previously Presented) A computer- implemented method of providing 
information to a user based upon contents of a document displayed to the user, the method 
comprising: 

accessing a first set of one or more content provider information objects (CPIOs); 

extracting a set of one or more user document information objects (UDIOs) from 
the document, the set of UDIOs comprising a first UDIO comprising information of a first type; 

identifying a plurality of selection techniques for determining degree of relevancy 
information for the first set of CPIOs, the first set of CPIOs comprising at least one information 
object comprising information of a second type that is different from the first type, the plurality 
of selection techniques comprising a first selection technique and a second technique, wherein 
the first technique compares contents of the UDIOs and the CPIOs, wherein the second 
technique determines relevancy of the UDIOs and the CPIOs to one or more concepts of interest 
to a user; 

for each selection technique in the plurality of selection techniques, applying the 
selection technique to generate degree of relevancy information for the CPIOs in the set of 
CPIOs, the degree of relevancy information indicating the relevancy of the CPIOs to the UDIOs 
in the set of UDIOs calculated using the selection technique; and 

selecting a second set of one or more CPIOs from the first set of CPIOs based 
upon the degree of relevancy information for the CPIOs calculated using the plurality of 
selection techniques, wherein information objects in the second set of CPIOs store information to 
be output to the user when the document is being displayed to the user. 

15. (Previously Presented) A computer-program product stored on a 
computer readable storage medium for providing information to a user based upon contents of a 
first document displayed to the user, the computer-program product comprising: 

code for identifying at least a first section of the first document; 
code for extracting a first set of one or more information objects from the first 
section of the first document, the first set of information objects comprising at least a first 
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information object comprising information of a first type, wherein the first information object is 
one of an audio object, a video object, or an image object; 

code for determining degree of relevancy information for a second set of one or 
more information objects, the degree of relevancy information indicating the relevancy of 
information objects in the second set of information objects to information objects in the first set 
of information objects, wherein the second set of information objects comprises at least one 
information object comprising information of a second type, wherein the second type is different 
from the first type, and wherein determining the degree of relevancy information comprises 
determining a degree of relevancy of the at least one information object to the first information 
object from the first set of information objects; 

code for selecting a third set of one or more information objects from information 
objects in the second set of information objects based upon the degree of relevancy information 
determined for information objects in the second set of information objects, wherein information 
objects in the third set of information objects store information to be output to the user when the 
first document is being displayed to the user. 

16. (Previously Presented) The computer-program product of claim 15 
wherein the first section of the first document corresponds to a section of the first document 
displayed to the user, wherein the section of the first document displayed to the user is less than 
the entire first document. 

1 7. (Previously Presented) The computer-program product of claim 1 5 
wherein the first section of the first document corresponds to the entire first document. 

18. (Previously Presented) The computer-program product of claim 1 5 
wherein the code for extracting the first set of information objects from the first section of the 
first document comprises: 

for each information object in the first set of information objects: 

code for identifying a type of the information object based upon contents 
of the information object; 
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code for determining a first content recognition technique based upon the 
type of the information object; and 

code for applying the first content recognition technique to the information 
object to determine information related to the contents of the information object. 

19. (Previously Presented) The computer-program product of claim 1 5 

wherein: 

the code for determining the degree of relevancy information for the second set of 
information obj ects comprises : 

code for identifying a plurality of selection techniques for determining the 
degree of relevancy information; and 

for each selection technique in the plurality of selection techniques, code 
for applying the selection technique to generate relevancy scores for the information objects in 
the second set of information objects, the relevancy scores indicating the relevancy of 
information objects in the second set of information objects to information objects in the first set 
of information objects calculated using the selection technique; and 

the code for selecting the third set of information objects comprises: 

code for selecting one or more information objects from the second set of 
information objects to be included in the third set of information objects based upon the 
relevancy scores for information objects in the second set of information objects calculated using 
the plurality of selection techniques. 

20. (Previously Presented) The computer-program product of claim 19 
wherein the code for selecting the one or more information objects from the second set of 
information objects to be included in the third set of information objects based upon the 
relevancy scores for information objects in the second set of information objects calculated using 
the plurality of selection techniques comprises: 

for each information object in the second set of information objects: 
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code for calculating an aggregate relevancy score for the information 
object by aggregating the relevancy scores generated for the information object by applying the 
plurality of selection techniques; and 

code for selecting the information object to be included in the third set of 
information objects if the aggregated relevancy score calculated for the information object is 
above a threshold value. 

2 1 . (Previously Presented) The computer-program product of claim 1 5 

wherein: 

the code for determining the degree of relevancy information for the second set of 
information objects comprises: 

code for identifying a first selection technique and a second selection 
technique for determining the degree of relevancy information; and 

code for applying the first selection technique to generate a first set of 
relevancy scores for information objects in the second set of information objects, the first set of 
relevancy scores indicating the relevancy of information objects in the second set of information 
objects to information objects in the first set of information objects calculated using the first 
selection technique; 

code for applying the second selection technique to generate a second set 
of relevancy scores for information objects in the second set of information objects, the second 
set of relevancy scores indicating the relevancy of information objects in the second set of 
information objects to information objects in the first set of information objects calculated using 
the second selection technique; and 

the code for selecting the third set of information objects comprises: 

code for selecting one or more information objects from the second set of 
information objects to be included in the third set of information objects based upon the first set 
of relevancy scores and the second set of relevancy scores. 
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22. (Previously Presented) The computer-program product of claim 21 
wherein the code for applying the first selection technique to generate the first set of relevancy 
scores comprises: 

code for determining a set of one or more concepts of interest to the user; 

code for determining relevancy of each information object in the first set of 
information objects to each concept in the set of concepts; 

code for determining relevancy of each information object in the second set of 
information objects to each concept in the set of concepts; and 

code for calculating the first set of relevancy scores based upon the relevancy of 
each information object in the first set of information objects to each concept in the set of 
concepts and based upon the relevancy of each information object in the second set of 
information objects to each concept in the set of concepts, wherein each relevancy score in the 
first set of relevancy scores indicates a degree of relevancy of an information object in the 
second set of information objects to an information object in the first set of information objects 
for a particular concept included in the set of concepts. 

23. (Previously Presented) The computer-program product of claim 2 1 
wherein the code for applying the second selection technique to generate the second set of 
relevancy scores comprises: 

for each information object in the first set of information objects: 

code for identifying a type of the information object based upon contents 
of the information object; 

code for determining a comparison technique based upon the type of the 
information object; and 

for each information objfect in the second set of information objects, code 
for applying the comparison technique to generate a relevancy score for the information object in 
the second set of information objects, the relevancy score indicating a degree of relevance of the 
information object in the second set of information objects to the information object in the first 
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set of information objects using the comparison technique determined based upon the type of the 
information object in the first set of information objects. 

24. (Previously Presented) The computer-program product of claim 15 further 
code for communicating the third set of information objects to a user system which is used to 
output information stored by information objects in the third set of information objects to the 
user. 

25. (Previously Presented) The computer-program product of claim 15 further 
comprising code for outputting information stored by information objects in the third set of 
information objects to the user in a predetermined area of an access program which is used to 
display the first document to the user. 

26. (Previously Presented) The computer-program product of claim 25 
wherein the access program is a web browser and the first document is a web page. 

27. (Previously Presented) The computer-program product of claim 15 further 

comprising: 

code for determining when a second document is displayed to the user instead of 
the first document; 

code for identifying at least a first section of the second document; 

code for extracting a fourth set of one or more information objects from the first 
section of the second document; 

code for determining new degree of relevancy information for the second set of 
information objects, the new degree of relevancy information indicating the relevancy of 
information objects in the second set of information objects to information objects in the fourth 
set of information objects; and 

code for selecting a fifth set of one or more information objects from information 
objects in the second set of information objects based upon the new degree of relevancy 
information determined for the second set of information objects, wherein information objects in 
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the fifth set of information objects store information to be output to the user when the second 
document is being displayed to the user. 

28. (Previously Presented) A computer-program product stored on a 
computer readable storage medium for providing information to a user based upon contents of a 
document displayed to the user, the computer-program product comprising: 

code for accessing a first set of one or more content provider information objects 

(CPIOs); 

code for extracting a set of one or more user document information objects 
(UDIOs) from the document, the set of UDIOs comprising a first UDIO comprising information 
of a first type; 

code for identifying a plurality of selection techniques for determining degree of 
relevancy information for the first set of CPIOs, the first set of CPIOs comprising at least one 
information object comprising information of a second type that is different from the first type, 
the plurality of selection techniques comprising a first selection technique and a second 
technique, wherein the first technique compares contents of the UDIOs and the CPIOs, wherein 
the second technique determines relevancy of the UDIOs and the CPIOs to one or more concepts 
of interest to a user; 

for each selection technique in the plurality of selection techniques, code for 
applying the selection technique to generate degree of relevancy information for the CPIOs in the 
set of CPIOs, the degree of relevancy information indicating the relevancy of the CPIOs to the 
UDIOs in the set of UDIOs calculated using the selection technique; and 

code for selecting a second set of one or more CPIOs from the first set of CPIOs 
based upon the degree of relevancy information for the CPIOs calculated using the plurality of 
selection techniques, wherein information objects in the second set of CPIOs store information to 
be output to the user when the document is being displayed to the user. 

29. (Previously Presented) A system for providing information to a user 
based upon contents of a first document displayed to the user, the system comprising: 

a processor; 
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a memory coupled to the processor, the memory configured to store a plurality of 
code modules for execution by the processor, the plurality of code modules comprising: 

a code module for identifying at least a first section of the first document; 

a code module for extracting a first set of one or more information objects 
from the first section of the first document, the first set of information objects comprising at least 
a first information object comprising information of a first type, wherein the first information 
object is one of an audio object, a video object, or an image object; 

a code module for determining degree of relevancy information for a 
second set of one or more information objects, the degree of relevancy information indicating the 
relevancy of information objects in the second set of information objects to information objects 
in the first set of information objects, wherein the second set of information objects comprises at 
least one information object comprising information of a second type, wherein the second type is 
different from the first type, and wherein determining the degree of relevancy information 
comprises determining a degree of relevancy of the at least one information object to the first 
information object from the first set of information objects; and 

a code module for selecting a third set of one or more information objects 
from information objects in the second set of information objects based upon the degree of 
relevancy information determined for information objects in the second set of information 
objects, wherein information objects in the third set of information objects store information to 
be output to the user when the first document is being displayed to the user. 

30. (Previously Presented) The system of claim 29 wherein the first section of 
the first document corresponds to a section of the first document displayed to the user, wherein 
the section of the first document displayed to the user is less than the entire first document. 

3 1 . (Previously Presented) The system of claim 29 wherein the first section of 
the first document corresponds to the entire first document. 
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32. (Previously Presented) The system of claim 29 wherein the code module 
for extracting the first set of information objects from the first section of the first document 
comprises: 

for each information object in the first set of information objects: 

a code module for identifying a type of the information object based upon 

contents of the information object; 

a code module for determining a first content recognition technique based 

upon the type of the information object; and 

a code module for applying the first content recognition technique to the 

information object to determine information related to the contents of the information object. 

33. (Previously Presented) The system of claim 29 wherein: 

the code module for determining the degree of relevancy information for the 
second set of information objects comprises: 

a code module for identifying a plurality of selection techniques for 
determining the degree of relevancy information; and 

for each selection technique in the plurality of selection techniques, a code 
module for applying the selection technique to generate relevancy scores for information objects 
in the second set of information objects, the relevancy scores indicating the relevancy of 
information objects in the second set of information objects to information objects in the first set 
of information objects calculated using the selection technique; and 

the code module for selecting the third set of information objects comprises: 

a code module for selecting one or more information objects from the 
second set of information objects to be included in the third set of information objects based 
upon the relevancy scores for information objects in the second set of information objects 
calculated using the plurality of selection techniques. 

34. (Previously Presented) The system of claim 33 wherein the code module 
for selecting the one or more information objects from the second set of information objects to be 
included in the third set of information objects based upon the relevancy scores for information 
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objects in the second set of information objects calculated using the plurality of selection 
techniques comprises: 

a code module for calculating an aggregate relevancy score for each information 
object in the second set of information objects by aggregating the relevancy scores generated for 
the information object by applying the plurality of selection techniques; and a code module for 
selecting an information object from the second set of information objects to be included in the 
third set of information objects if the aggregated relevancy score calculated for the information 
object is above a threshold value. 

35. (Previously Presented) The system of claim 29 wherein: 
the code module for determining the degree of relevancy information for the 
second set of information objects comprises: 

a code module for identifying a first selection technique and a second 
selection technique for determining the degree of relevancy information; and 

a code module for applying the first selection technique to generate a first 
set of relevancy scores for information objects in the second set of information objects, the first 
set of relevancy scores indicating the relevancy of information objects in the second set of 
information objects to information objects in the first set of information objects calculated using 
the first selection technique; 

a code module for applying the second selection technique to generate a 
second set of relevancy scores for information objects in the second set of information objects, 
the second set of relevancy scores indicating the relevancy of information objects in the second 
set of information objects to information objects in the first set of information objects calculated 
using the second selection technique; and 

the code module for selecting the third set of information objects comprises: 

a code module for selecting one or more information objects from the 
second set of information objects to be included in the third set of information objects based 
upon the first set of relevancy scores and the second set of relevancy scores. 
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36. (Previously Presented) The system of claim 35 wherein the code module 
for applying the first selection technique to generate the first set of relevancy scores comprises: 

a code module for determining a set of one or more concepts of interest to the 

user; 

a code module for determining relevancy of each information object in the first 
set of information objects to each concept in the set of concepts; 

a code module for determining relevancy of each information object in the second 
set of information objects to each concept in the set of concepts; and 

a code module for calculating the first set of relevancy scores based upon the 
relevancy of each information object in the first set of information objects to each concept in the 
set of concepts and based upon the relevancy of each information object in the second set of 
information objects to each concept in the set of concepts, wherein each relevancy score in the 
first set of relevancy scores indicates a degree of relevancy of an information object in the 
second set of information objects to an information object in the first set of information objects 
for a particular concept included in the set of concepts. 

37. (Previously Presented) The system of claim 35 wherein the code module 
for applying the second selection technique to generate the second set of relevancy scores 
comprises: 

for each information object in the first set of information objects: 

a code module for identifying a type of the information object based upon 
contents of the information object; 

a code module for determining a comparison technique based upon the 
type of the information object; and 

for each information object in the second set of information objects, a 
code module for applying the comparison technique to generate a relevancy score for the 
information object in the second set of information objects, the relevancy score indicating a 
degree of relevance of the information object in the second set of information objects to the 
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information object in the first set of information objects using the comparison technique 
determined based upon the type of the information object in the first set of information objects. 

38. (Previously Presented) The system of claim 29 wherein the plurality of 
code modules further comprises: 

a code module for communicating the third set of information objects to a user 
system which is used to output information stored by information objects in the third set of 
information objects to the user. 

39. (Previously Presented) The system of claim 29 wherein the first document 
is displayed to the user using an access program and information stored by information objects in 
the third set of information objects is output to the user in a predetermined area of the access 
program. 

40. (Previously Presented) The system of claim 39 wherein the access 
program is a web browser and the first document is a web page. 

41. (Previously Presented) The system of claim 29 wherein the plurality of 
code modules further comprises: 

a code module for determining when a second document is displayed to the user 
instead of the first document; 

a code module for identifying at least a first section of the second document; 

a code module for extracting a fourth set of one or more information objects from 
the first section of the second document; 

a code module for determining new degree of relevancy information for the 
second set of information objects, the new degree of relevancy information indicating the 
relevancy of information objects in the second set of information objects to information objects 
in the fourth set of information objects; and 

a code module for selecting a fifth set of one or more information objects from 
information objects in the second set of information objects based upon the new degree of 
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relevancy information determined for the second set of information objects, wherein information 
objects in the fifth set of information objects store information to be output to the user when the 
second document is being displayed to the user. 

42. (Previously Presented) A system for providing information to a user 
based upon contents of a document displayed to the user, the system comprising: 
a user system displaying the first document to the user; and 
a server system coupled to the user system; 
wherein the server system is configured to: 

access a first set of one or more content provider information objects 

(CPIOs); 

extract a set of one or more user document information objects (UDIOs) 
from the document, the set of UDIOs comprising a first UDIO comprising information of a first 
type; 

identify a plurality of selection techniques for determining degree of 
relevancy information for the first set of CPIOs, the first set of CPIOs comprising at least one 
information object comprising information of a second type that is different from the first type, 
the plurality of selection techniques comprising a first selection technique and a second 
technique, wherein the first technique compares contents of the UDIOs and the CPIOs, wherein 
the second technique determines relevancy of the UDIOs and the CPIOs to one or more concepts 
of interest to a user; 

for each selection technique in the plurality of selection techniques, apply 
the selection technique to generate degree of relevancy information for the CPIOs in the set of 
CPIOs, the degree of relevancy information indicating the relevancy of the CPIOs to the UDIOs 
in the set of UDIOs calculated using the selection technique; and 

select a second set of one or more CPIOs from the first set of CPIOs based 
upon the degree of relevancy information for the CPIOs calculated using the plurality of 
selection techniques; and 
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wherein the user system is configured to output information stored by the second 
set of CPIOs to the user. 

43. (Previously Presented) The system of claim 42 wherein at least one CPIO 
included in the first set of CPIOs is provided by a content provider system coupled to the server 
system. 

44. (Previously Presented) The system of claim 42 wherein the first set of 
CPIOs comprises a first CPIO and a second CPIO, wherein the first CPIO is provided by a first 
content provider system coupled to the server system and the second CPIO is provided by a 
second content provider system coupled to the server system. 
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